The present invention, generally, relates to network communication methods, systems and computer program products and, more particularly, to systems, methods and computer program products for performance testing of computer networks.
Companies are often dependent on mission-critical network applications to stay productive and competitive. To achieve this, information technology (IT) organizations preferably provide reliable application performance on a 24-hour, 7-day-a-week basis. In this demanding environment, frequent network expansion and day-to-day fire fighting often leave little time for IT managers to manage network performance proactively. Accordingly, IT managers typically rely on some form of system management and/or network management tools to help automate performance management tasks to increase the efficiency of the IT staffs. Even with these tools, IT staffs are typically required to commit resources to integrating and customizing the tools to work in the heterogeneous network environments which may include, for example, hardware from a variety of vendors, executing a variety of operating systems and communication protocols and supporting a variety of application programs for different end user requirements.
Various known system management products track specific information, such as the CPU utilization on a server, server paging and disk access activity, or client application response time. This information may be useful when solving problems on a network. These products can generally be categorized as passive system and application monitors. This category typically is implemented as software agent technologies that reside on the client or server computers. They generally passively monitor live application transactions and monitor resource utilization. Products in this category include Patrol from BMC Software, Inc., FirstSense Agent from FirstSense Software, Inc., S3 from NextPoint Networks, Inc., VitalSuite from INS, Luminate Software Corp., and Envive Corp. As they are passive application monitors, they typically support only specific application programs. For example, Luminate Software and Envive support the SAP R/3 application, NextPoint Networks and BMC Software support the Oracle database. Their measurements are neither consistent nor repeatable, as a user""s interaction with a given application varies over time. moreover, they are typically not suited to detecting system slowdowns or failures from the perspective of an end user.
Another category of of system management tool is active application monitors. These are products that generally measure performance by actively emulating application transactions. These transactions are often referred to as xe2x80x9csyntheticxe2x80x9d transactions. Products in this category include VeriServ from Response Networks, Inc. and SLM from Jyra Research Inc. VeriServ allows an operator to define the types of applications to be monitored, times and days, and the end user locations from which the transactions are to originate. The operator may also choose to define alarm thresholds. Agents installed at the end user location monitor actual sample application transactions to measure performance of the applications operating over the network environment. VeriServ automatically tests applications at a fixed interval. SLM provides the flexibility for the user to schedule synthetic transactions for any interval from 5 minutes to a year. However, as these approaches are also typically directed to a particular application and require that the applications be installed and operating to generate network traffic, they generally only address simple web and database transactions. Also, any new or custom applications may require extensive configuration by the users to allow the tester to interact with the applications.
Another approach to passive monitoring is directed to the network rather than the overall system. On the network side, element managers or passive network monitors are known which may address a specific segment or device on the network. Element managers are generally software designed to manage specific groups of devices, such as routers and switches. Passive network monitors are typically a combination of hardware and software that may, for example, monitor network traffic at the link layer or at the infrastructure devices. Products falling in this category include remote monitor (RMON) probes from NetScout Systems, Inc., Sniffer from Network Associates, NetMetrix from Hewlett-Packard, Application Expert from Optimal Networks Corp., EcoSCOPE from Compuware Corp., and Visual OnRamp from Visual Networks, Inc. These network management tools typically provide information such as packet loss, bit rates, and network utilization. This type of information may be helpful in fixing a network problem after the problem has been identified. However, as with the passive system monitors, these tools generally do not reflect network performance as experienced by a user. These tools are passive, in that they generally watch the network traffic which traverses a network segment or link, rather than actively creating traffic.
Passive network monitors sometimes include a basic scheduler to collect sample data from their data sources. A basic scheduler generally merely specifies the frequency (e.g., once every 15 minutes) at which the management console of the monitor should collect data from the data sources. Passive monitors are limited in that they are typically expensive to scale, only see traffic that is on the network at the time and do not recognize new or custom application traffic. Furthermore, it is difficult, if not impossible, to get a total end-to-end view of network performance from the sum of the individual elements as observed by passive monitors.
All of these approaches are limited in their ability to track network performance on a continuing basis in a manner which will reflect that performance from an end user perspective across a variety of hardware, operating systems and end user specific application programs while supporting multiple network protocols under actual operating conditions. This creates particular problems as networks are typically not static as new hardware and software application programs may be periodically added thereby changing the traffic characteristics on the network and the end user""s experience of network performance. This changing environment also makes it difficult if not impossible for any of these known tools to provide repeatable measurements over time for use in assessing changes in network performance while improving the efficiency of IT staffs rather than increasing their workload. It is increasingly important to analyze the actual performance of the network to be tested without the constraints and limitations of these existing tools. It would also be beneficial to provide for automated evaluation of the acceptability of monitored performance of the network.
Accordingly, it is an object of the present invention to provide network performance testing methods, systems and computer program products which can provide network managers the ability to test the performance of a network in an ongoing manner.
It is a further object of the present invention to provide such network performance methods, systems and computer program products which can have flexibility in how and when network performance testing is executed.
It is a further object of the present invention to provide such network performance methods, systems and computer program products which can determine if performance is acceptable.
These and other objects are provided by systems, methods and computer program products which test network performance by defining test schedules including test protocols to be implemented and when the protocols should be executed for a plurality of defined connections on a network. A connection may be defined between two endpoint nodes on the network. At times specified in the test schedule, the endpoint node pair executes the test protocol and measures the performance of the network connection between the two nodes without requiring any involvement of application software which may or may not be installed on the computer hardware supporting the endpoint node. The test protocol may define the type of network layer protocol to utilize (for example, TCP), and the test script or scripts to be communicated using the appropriate network protocol stack on the computer hardware supporting the endpoint node. The schedule may be provided with an expiration date and a console node is provided for distribution of test schedules, monitoring of availability of endpoint nodes and receipt of measured performance measurements for reporting to a network manager. In further aspects of the present invention, auto-thresholding and coordination of interrelated but asynchronous tasks executing at the console node are provided.
The present invention provides a unique approach to network testing which may provide consistent and repeatable measurements. IT staffs are provided the capability to define schedules, distribute complex tests to endpoints to test connections without regard to the end user application programs available at particular endpoints, and automate periodic performance testing. Network management may thereby be separated from the separate and often challenging problem of application program management in a network environment. Network test results may encompass an end-to-end view and may further break network performance analysis down into its components, such as client, server, application, and network time, to potentially quickly and accurately isolate problems. This, in turn, may reduce downtime and provide better service to end users while increasing the efficiency of IT staffs.
In one embodiment of the present, a method is provided for testing the performance of a communications network including a plurality of endpoint nodes. A console node residing on a computer is added to the network. A connection associated with at least one of the endpoint nodes and a test schedule for the connection are defined at the console node and the test schedule is communicated to the associated endpoint node from the console node. Network performance test results are received at the console node from the endpoint node based on the test schedules. The connection definition may include defining a network communication test protocol based on a type of application communication traffic expected on the network and may further may include defining an endpoint node specific test protocol between the at least one endpoint node and an associated second endpoint node to simulate communications traffic therebetween to be tested.
In a further embodiment of the present invention, a performance threshold may be defined for the connection and an action may be defined to be taken responsive to a detected violation of the performance threshold. The performance threshold may be an auto-threshold wherein the performance threshold is determined based on determinations of a selected performance characteristic from obtained network performance test results. In one embodiment of the auto-thresholding aspect of the present invention, the auto-threshold provides a first detection criteria establishing the performance threshold for the connection if the connection has a first standard deviation for the selected performance characteristic and a second detection criteria establishing the performance threshold for the connection if the connection has a second standard deviation for the selected performance characteristic, the first standard deviation exceeding a selected standard deviation and the second standard deviation being less than the selected standard deviation.
In another embodiment of the present invention, a plurality of connections associated with the plurality of endpoint nodes are defined. Each of the plurality of connections is associated with a pair of the plurality of endpoint nodes and defines an endpoint node specific test protocol between the pair of the plurality of endpoint nodes to simulate communications traffic therebetween to be tested. A test schedule is defined for each of the plurality of connections, each test schedule including a start time for initiating execution of the associated endpoint node specific test protocol and a repeat schedule for re-initiating execution of the associated endpoint node specific test protocol. The test schedule may also include an expiration time for terminating re-initiating execution of the associated endpoint node specific test protocol. The test schedules, including the endpoint node specific test protocols associated with each of the test schedules, are then transmitted to at least one of the associated pair of the plurality of endpoint nodes for each test schedule.
In a further embodiment of the present invention, network performance results are received at the console node over the network from at least one of the associated pair of the plurality of endpoint nodes based on measurements from execution of the associated endpoint node specific test protocol. The network performance results may be received at time intervals based on the associated test schedule of the connection associated with the at least one of the associated pair of the plurality of endpoint nodes. The console node may also re-connect with the at least one of the associated pair of the plurality of endpoint nodes if network performance results are not received from the at least one of the associated pair of the plurality of endpoint nodes within a criteria time interval after expiration of one of the time intervals at which results were expected. An error may be indicated by the console node if it is unable to re-connect with the at least one of the associated pair of the plurality of endpoint nodes after a selected number of attempts to re-connect.
In one embodiment of the present invention, one of the endpoint nodes of a connection receives the test schedule and executes the endpoint node specific test protocol over the network between the at least one endpoint node and the associated second endpoint node at a start time for initiating execution specified in the test schedule. The endpoint node further monitors the performance of the network while the endpoint node specific test protocol is executed to obtain the network performance test results and communicates the network performance test results to the console node over the network. The executing, monitoring and communicating operations are repeated at a repeat time for re-initiating execution specified in the test schedule. The network performance test results may be communicated at a selected time following completion of the monitoring step.
While the invention has been described above primarily with reference to the method aspects of the present invention, it is to be understood that the present invention may also be embodied as systems and computer program products.
Accordingly, the systems, methods and computer program products of the present invention can provide a capability to test networks on an ongoing basis, under selectable operating conditions not provided by existing network performance testing tools. Test protocols for different connections may be designed to reflect expected operating conditions including the flexibility to provide distinct endpoint specific test protocols to different endpoints or endpoint node pairs associated with a connection which then execute their individual protocols pursuant to the provided test schedules. Performance results from each execution of a test protocol under a test schedule may be reported back to a console node for analysis. No specialized hardware is required and tests can be readily expanded without a corresponding increase in hardware costs. Network testing schedules can be established and executed without further operator intervention. In addition, performance thresholds may be implemented with actions triggered when thresholds are violated. The present invention thereby provides network managers with an improved tool for designing and managing complex computer networks handling a variety of network protocols, hardware and applications traffic.